Computational Intelligence in Financial Forecasting and Agent-Based Modeling: Applications of Genetic Programming and Self-Organizing Maps

This thesis focuses on applications of Computational Intelligence techniques to Finance and Economics. First of all, we build upon a Genetic Programming (GP)-based financial forecasting tool called Evolutionary Dynamic Data Investment Evaluator (EDDIE), which was developed, and reported on in the past, by researchers at the University of Essex. The novelty of the new version we present, which we call EDDIE 8, is its extended grammar, which allows the GP to search in the space of the technical indicators in order to form its trees. In this way, EDDIE 8 is not constrained into using pre-specified indicators, but it is left up to the GP to choose the optimal ones. Results show that, thanks to the new grammar, new and improved solutions can be found by EDDIE 8. Furthermore, we present work on the Market Fraction Hypothesis (MFH). This hypothesis is based on observations in the literature about the fraction dynamics of the trading strategy types that exist in financial markets. However, these observations have never been formalized before, nor have they been tested under real data. We therefore first formalize the hypothesis, and then propose a model, which uses a twostep approach, for testing the hypothesis. This approach consists of a rule-inference step and a rule-clustering step. We employ GP as the rule inference engine, and apply Self-Organizing Maps (SOMs) to cluster the inferred rules. After running experiments on real datasets, we are able to obtain valuable information about the fraction dynamics of trading strategy types, and their long and short term behavior. Finally, we present work on the Dinosaur Hypothesis (DH), which states that the behavior of financial markets constantly changes and that the population of trading strategies continually co-evolves with their respective market. To the best of our knowledge, this observation has only been made and tested under artificial datasets, but not with real data. We formalize this hypothesis by presenting its main constituents. We also test it with empirical datasets, where we again use a GP system to infer rules and SOM for clustering purposes. Results show that for the majority of the datasets tested, the DH is supported. Thus this indicates that markets have non-stationary behavior and that strategies cannot remain effective unless they continually adapt to the changes happening in the market. List of Publications REFEREED JOURNALS • Kampouridis, M., Chen, S.-H., Tsang, E.: “Market Fraction Hypothesis: A Proposed Test”, International Review of Financial Analysis, special issue on Complexity and Non-Linearities in Financial Markets: Perspectives from Econophysics, forthcoming. (Chapter 7) (Kampouridis et al, 2011b) • Kampouridis, M., Chen, S.-H., Tsang, E.: “Microstructure Dynamics and AgentBased Financial Markets: Can Dinosaurs Return?”, Advances in Complex Systems. (Chapter 8) REFEREED BOOK CHAPTERS • Kampouridis, M., Chen, S.-H., Tsang, E.,“The Market Fraction Hypothesis under different GP algorithms”, Information Systems for Global Financial Markets: Emerging Developments and Effects, IGI Global, 2011, (Invited book chapter) forthcoming. (Chapter 7) (Kampouridis et al, 2011c) • Kampouridis, M., Chen, S.-H., Tsang, E.: “Market Microstructure: A SelfOrganizing Map Approach for Investigating Behavior Dynamics under an Evolutionary Environment”, in Brabazon, A., O’Neil, A. (Eds.), Natural Computing in Computational Finance, Volume 4, Studies in Computational Intelligence Series, Springer, 2011 (Invited book chapter) forthcoming. (Chapter 8) (Kampouridis et al, 2011d) REFEREED PAPERS IN CONFERENCE PROCEEDINGS • Kampouridis, M., Tsang, E.: “EDDIE for Investment Opportunities Forecasting: Extending the Search Space of the GP”, In Proceedings of the IEEE Congress on Evolutionary Computation, p. 2019–2026, Barcelona, Spain, 2010. (Chapter 6) (Kampouridis and Tsang, 2010) • Kampouridis, M., Tsang, E.: “Using Hyperheuristics under a GP framework for Financial Forecasting”, in Carlos A. Coello Coello (Ed.) Learning and Intelligent Optimization – LION 5, Lecture Notes in Computer Science, Rome, Italy, 2011. (Extension of Chapter 6) (Kampouridis and Tsang, 2011) • Chen, S.-H., Kampouridis, M., Tsang, E.,“Microstructure Dynamics and AgentBased Financial Markets”, In T. Bosse, A. Geller, and C.M. Jonker (Eds.): MultiAgent-Based Simulation XI, 11th InternationalWorkshop, Revised Papers, LNAI 6532, Springer, Heidelberg, pp. 121-135. (Chapter 7) (Chen et al, 2011) • Chen, S.-H., Kampouridis, M., Tsang, E.: “Microstructure Dynamics and AgentBased Financial Markets”, In Tibor Bosse, Armando Geller, Catholijn M. Jonker (eds.), Proceedings of the 11th International Workshop on Multi-Agent-Based Simulation (MABS), p. 117–128, Toronto, 11 May 2010. (Chapter 7) (Chen et al, 2010) • Kampouridis, M., Tsang, E. “Testing the dinosaur hypothesis under empirical datasets”. In R. Schaefer et al., editor, Parallel Problem Solving in Nature (PPSN) XI, Part II, LNCS 6239, p. 199–208, Heidelberg, 2010. Springer. (Chapter 8) (Kampouridis et al, 2010c) • Kampouridis, M., Chen, S.-H., Tsang, E.,“Market Microstructure: Can Dinosaurs Return? A Self-Organizing Map Approach under an Evolutionary Framework”, in C. Di Chio et al. (Eds.): EvoApplications 2011, Part II, LNCS 6625, pp. 91–100. Springer, Heidelberg (2011). (Chapter 8) (Kampouridis et al, 2011e) • Kampouridis, M., Chen, S.-H., Tsang, E.,“Investigating the Effect of Different GP Algorithms on the Non-Stationary Behavior of Financial Markets”, IEEE Symposium on Computational Intelligence for Financial Engineering & Economics, 11-15 April 2011, Paris, France. (Chapter 8) (Kampouridis et al, 2011a) • Kampouridis, M., Chen, S.-H., Tsang, E.: “Testing the Dinosaur Hypothesis Under Different GP Algorithms”, In Proceedings of the UK Computational Intelligence (UKCI) Workshop, Essex, IEEE Xplore, 2010. (Chapter 8) (Kampouridis et al, 2010b) EXTENDED ABSTRACTS • Kampouridis, M., Tsang, E.: “Hyper-Heuristics for Investment Opportunities Forecasting”, In Computational Management Science (CMS), London, 2008 (Chapter 6) (Kampouridis et al, 2008) • Kampouridis, M., Chen, S.-H., Tsang, E.: The Market Fraction Hypothesis: A proposed test, In Proceedings of the Econophysics Colloquium 2010, Taipei, 2010. (Chapter 7) (Kampouridis et al, 2010a) (earlier version also presented in Complex’09 and CMS 2009-see below) • Kampouridis, M., Chen S.-H., Tsang, E.: “Market fraction hypothesis: A proposed test”, In Proceedings of the 9th Asia-Pacific Complex Systems Conference, Tokyo, 2009 (Chapter 7) (Kampouridis et al, 2009a) • Kampouridis, M., Chen S.-H., Tsang, E.: “Market fraction hypothesis: A proposed test”, In Computational Management Science (CMS), Geneva, 2009 (Chapter 7) (Kampouridis et al, 2009b) TECHNICAL REPORTS • Kampouridis, M., Tsang, E.: “EDDIE on Artificial Dataset”, Technical Report CES492, University of Essex. (Chapter 6) (Kampouridis and Tsang, 2009) • Kampouridis, M., Chen S.-H., Tsang, E.: “A summary for the Brock and Hommes ‘Heterogeneous beliefs and routes to chaos in a simple asset pricing model’ 1998 JEDC paper”, Technical Report CES497, University of Essex. (Chapter 7) (Kampouridis et al, 2009c) List of Works Under Review • Kampouridis, M., Tsang, E.: “Investment Opportunities Forecasting: Extending the Grammar of a GP-based Tool”, International Journal of Computational Intelligence Systems. (Chapter 6) • Kampouridis, M., Alsheddy, A.: “On the investigation of hyper-heuristics on a financial forecasting problem”, Annals of Mathematics and Artificial Intelligence, Springer. (Invited paper) (Extension of Chapter 6) • Kampouridis, M., Glover, T., Rais Shaghaghi, A., Tsang, E.: “Deciding the optimal roll-out plan for the deployment of fiber optic networks”, Engineering Optimization.